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T he term ’^‘data-driven deci- 
sion making” has become 
ubiquitous in educarion, and 
it seems to be most often 
discussed with reference to policy deci- 
sions related to reporting requirements 
and accountability. What deserves at 
least equal attention is what would enable 
teachers and administrators to use student 
assessment results to learn about stu- 


dents’ skills and about the eHectiveness of 
instnicrion — and then to apply that learn- 
ing to instructional improvements. As a 
partner in the National Research Center 
for Career and Technical Education 
(NRCCTE) funded by the Depart- 
ment of Education, Office of Adult and 
Vocational Education (OVAE), NOCTI 
detcrinincd to give the issue attention by 
conducting a research survey’. 


NOCTI has a long history, well over 
four decades, of providing comprehensive 
occupational skill assessments to measure 
student achievement. One of the oi^aniza- 
tion^ core beliefs is that compreliensiv'e 
career and technical education (CTE) stu- 
dent assessment b one of the most effective 
tools for increasing student achievement 
and motivation througli educators’ use of 
data for instructional purposes. This study 
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Figure 2: Exomples of Changes Made With IndividyDl Students Bused on Data 
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investigated how secondary GTE 
educators, both leactiers and administra- 
tors, use technical assessment: data to im- 
prove program ctirriailum and to identify 
individual /group mstrudjon needs. Of 
particular interest in developing the 
sur vey was to ask hcHv they have lear ned 
to make use of the data; what specilk 
types of professional development were 
provided, ifaiiy; their perception oft lie ef- 


fectiveness of this training; and what types 
of professional de\'elopmeiu tlrey wotdd 
consider most effective for tlie future. 

Tlie study design involved e-mailing 
the survey to a purposefid sample of CTE 
educators in five states (Illinois, Missouri, 
Oklalx>ma, Pennsylvania and Virginia), 
The survey was dbtribtited thanks to 
collaboration with the five state direc- 
ton of CTE, through the directors of a 


random sample of the secondary^ CTE 
center and the career tech coordinator of 
a random sample of comprehensive higli 
schools, and to ail the CTE instructors 
in tl»e same foursdeclcd occupational 
clusters (Business Eklucation, Construc- 
tion, Health Sdenccj and Manufacturing)^ 
Tlie selection of four programs was made 
so that the data for those programs coiJd 
be compared across sites. 

The criteria for selecrioii were intended 
to result in a set of programs as repiesen- 
tative as possible of the whole of CTE, 
and as iion-duplkative as possible of vari- 
ables such as capital intensity, program 
outcome with regard to certification, 
student gender preponderance, cluster, 
and use of technology. The overriding cri- 
terion was that they be commonly enough 
found among schools that the set would 
have a high probability of providing a 
large number of schools for the sampling 
frame. 

In Older to distance itself from the 
sur vey administration, NOCTI subcon- 
tracted that process to the Pennsylva- 
nia State Univeisity Survey Research 
Center (PSUSRC). The value ofhaving 
the survtiy administered by an external 
and impartial organization was weighed 
against the loss of time working with 
another entity, hi addition to the stirv^s, 
a variety of small case study interviews 
were conducted w'lth individual schools in 
the five states that were ktiown to be using 
assessment data to make instruclnoiial 
impiwemems. The information gleaned 
from these case studies will be included 
in the project final report, w-hicli will be 
available after OVAE clearance. 

Data Anolyses and Findings 

The data set w^as review'ed from a descrip- 
tive standpoint {e.g., response frequencies), 
the IjypotJieses have been ex ami tied as 
mucli as possible, and several potential 
trends r elated to the study hypotheses 
could be seen in the data set. Of the 
respondents, the vast majority indicated 
that their CTE students took end-of- 
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program techtiical assessments* The 
results of these assessments were used for 
A variety of purposes^ the most common 
of which was maintaining a continuous 
improvement process > making improve- 
ments to programs in areas in which 
scores were weak, and reporting to out- 
side boriies. 

A primary objective of this study was 
to investigate ilte extent to which CTE 
educators used technical assessment 
data to inform instructional improve- 
ment and the sources oftlieir knowledge 
that enables them to do so* A majority 
of the responders indicated that they do 
use technical assosinent data to make 


tors who responded, 3U percent had not 
received such professional development; 
among teachers, 35*9 percent had not 
received professional development. Of 
those wlio liad received such professbnaJ 
development, most seemed to feel that 
the trajiiing contained information they 
needed at an appropriate level. A third 
objective of this study was to determine 
how educatoif rated the types of profes- 
sional development and what forms of 
professional development on use of assess- 
ment data they felt would be most useful. 
The lesults were mixed, but they did 
seem to indicate that a inixture onbrinal 
training and practical foliow-up would be 


Hgure 1: Desired Professional 
Development Topics 
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. .states must build data systems that can track student performance 
from one year to the next, from one school to another, so that those 
students and their parents know when they are making progress and 
when they need extra attention. This information must also be put in 
the hands of educators so they can use it to improve instruction." 


instructional decisions* Of those who did 
not use the data to make improvements, 
most expressed a belief that they should 
be using them. Figures I and 2 show 
some of the most commoti instructional 
changes educators made ba^d on data. 

When asked how they learned to use 
data 10 make instructional decisions, the 
most common mode among administra- 
tors was during teacher or administra- 
tive training, followed by self-taught. For 
teachers, the most common method was 
self-taught, followed by teacher training 
and professional devdopmeiu, A second 
objective of this study was to examine 
the types of piofessional developmem 
CTE educators have received related to 
the use of data and how those offerings 
were perceiv^ed. Among tlie administ ra- 


the most helpful. Respondents were also 
asked about areas for which they did not 
have professional de%^dopment but wished 
it were available. Figure I shows the most 
frequently cited topics. 

Respondents were also asked whom 
they would prefer as a deliv'ery agent for 
professional development in the use of 
assessment data for data-based decision 
making. The top three choices for teach- 
ers were a knowledgeable teacher or ()eer; 
a consiiitant; and a representative from 
a professional testing organization. The 
top four choices for administrator were a 
school, district, or state data specialist; a 
knowledgeable teacher or peer; a consul- 
tant; and a representative from a profes- 
sional testing organization. 

Tlie study posed several research 


hypotheses, the first of w'hich w-as that 
educators who know more about test data 
interpreiaiion will tend to use the data (or 
instructional improvements than those 
who know less. As indicated above, many 
respondents indicated that they do use 
data for the purpose of making instruc- 
tional i mprove me nl5 5 although many 
administrators also indicated that they felt 
their teachers had a need for additional 
training on the use of assessment data for 
data-based decision making. 

Overall, both teachers and ad min h- 
tratois indicated that they had a positive 
perception of the value ofteclinical skill 
assessments, although administrators had 
a slightly more positive view than did 
teachers. A second researdi hypothesis in 
the study was that those %vho use test data 
for program improvement perceive an 
impact from the changes. The descriptive 
findings show qualitative support for this 
hypothesis. Some of the types of iiistruc- 
tional changes educators made are dis- 
cussed above. When asked if they found 
those changes to be effective, t lie majority 
of respondents indicated that they did see 
them as effective. The rest of those who 
responded to the question indicated that 
they were unsure. 

A thitxl research hypothesis was that 
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those who use test data for program iiiv 
provement have had professional devel- 
opment on the topic. Various responses 
to several questions in the descriptive 
findings indicate that educators see the 
value of using data to make instructional 
improvements^ and that professional 
development can be helpful in learning 
how to interpret and apply data most 
effectively. 

Next Steps 

In addition to the objectives and research 
hypotheses discussed above, the survey 
also solicited information on a variety of 
other factors that were deter inincti to be 
potentially helpful in addressing a fourth 
ot^ective to be completed next — that 

ofereating a professional development 
system geared toward improving educator 
undemanding of the use of technical as- 
sesstiietii data, and increasing the use and 


effectiveness of such data in making in- 
structional improvement a. Findings from 
the survey will be combined with insights 
gleaned from die literature to provide 
a basis for the creation of professional 
des^lopment that can be pilot-tested and 
later provided in response to technical 
assistance requests fmm states. 

Signifkaiue 

U.S. Secretary of Education Arne 
Duncan, In testimony before the House 
Education and Labor Committee, said, 
stares must build data systems ihai can 
track student performance from one year 
to the next, from one school to another, 
so that those students and their p rents 
know w hen they are making progress 
and w'ben they need extra atiention. This 
information must also be put in the hands 
of educators so diey can use it to improve 
instructioif' (Edgov, 2009). This study 


was especially timely because this is the 
first time that, through tlie 2006 reautho- 
rizatioii of the Carl D. Perkins Career and 
TedinicaJ Education Act (Perkiiis iV), 
states must report on tlie technical skill at- 
tainment of student concentrators in GTE 
programs, li is time to help educators 
learn from data so that they can improve 
student performance. D 
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• Ability to design furniture to maximize available space and square footage 

- Wire management system conceals wires and cables while allowing easy access 

• Lifetime Warranty means low maintenance and low total cost of ownership 

• Furniturecontribytes to a healthy indoor environment 



Call usorvisrt usor> the web to find your local representative 
800-67B-5550 - www.interiorcancepts.com 
Visit us at the ACTE Expo In Nashvtiie, Booth #2231 


^Interior Concepts' 

^ solutions that fit 


w w w . o€ tec n M ne. o 


NOVIMIIR/I^ICtMIEIt 2069 Technlquei 37 










